WORKLOAD MANAGEMENT SYSTEM FOR NURSING 

Interrater Reliability (IRR) Testing 
For use of this form, see FM 8-501 ; the proponent agency is the OTSG. 


SEE INSTRUCTIONS 
ON REVERSE BEFORE 
COMPLETING. 


1 . Hospital 


2. Unit 


3. Date 


4. IRR Rater 



5. Sample Size Selection Table Unit Sample Size 



<5 


□ all patients 
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6 


>21 and <30 
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>31 and <40 
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>41 
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9 



6. Catagory Agreement f (# Agreed) x100% f x100%= % Agreement 

e(#Sampled) e 

Provide the MEPRS Office with this IRR Category Agreement score and the date. 
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7. Critical Indicator Agreement 
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g. Overall Totals 

8. Notes on Differences. 









DA FORM 5391 -R, JUN 1990 



EDITION OF FEB 1985 IS OBSOLETE. 
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INSTRUCTIONS FOR COMPLETION OF DA FORM 5391-R 



1 . Hospital: Name and location of hospital. 

2. Unit: The alpha/numeric description of the nursing unit. 

3. Date: The date of the IRR test. 

4. IRR Rater: The signature of the person conducting the test is necessary for accountability. 

5. Sample Size Selection Table: Select the sample size based on the unit census. Use the 
table provided. 

6. Category Agreement: Compute the percentage of category agreement by dividing the 

total number of category agreements (6f) by the total number of patients in the sample (6e). 
Convert this number to a percentage by multiplying by 1 00. 

a. Record a patient identification number (the number used to randomly select the patient) 
for each patient in the test sample. 

b. Enter the acuity category for each patient determined by the unit nurse. 

c. Enter the acuity category for each patient determined by the IRR rater. 

d. Place an "X" mark for each patient in which both the original category and the IRR 
category are the same. 

e. Enter the total sample size. 

f. Enter the total number of category agreements. 

7. Critical Indicator Agreement: 

a. Record a patient identification number (the number used to randomly select the patient) 
for each patient in the test sample. 

b. Factor: Record three factors to be analyzed, such as vital signs, ADL, and feeding. For 
each patient in the sample, compare each critical indicator and points ascribed by the unit staff 
member and the IRR rater. Identify the number of matches, i.e., the number of times both 
people either selected or chose not to select a critical indicator. Record this number for each 
patient and each category in the appropriate space. 

c. #: Count the number of critical indicators for each of the selected factors and place in 
the # column. 

d. Total Agreed: Sum the number of exact matches for all patients in the sample and enter 
the total. 

e. Max Poss: For each listed factor, multiply the number of critical indicators in that factor 
(located in 7c # column) by the number of patients in the sample (6e) and enter the total. This 
indicates the maximum possible agreement count. 

f. % Agreed: For each listed factor, divide the "Total Agreed" (7d) by the "Max Poss" 

(7e). Convert this number to a percentage by multiplying by 1 00. 

g. Overall Totals: Sum columns 7d and 7e, and determine the overall % agreement in the 
same manner as 7f above. 

8. Document differences, especially those which may have resulted in a different category 
classification. 



REVERSE OF DA FORM 5391-R, JUN 1990 
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